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1. Introduction 


1.1 Purpose 

Developments in current and future information, decision making, and control systems will 
enable information creation and distribution to achieve extraordinary feats and challenge 
operators to attain new levels of performance. While research and technology address issues of 
information requirements, distribution, and decision support, the problem of information 
overload and the challenge of optimal infonnation display remain enduring obstacles. How can 
future information displays be designed to optimize information perception, interpretation, and 
decision making? Decision making rests on more than effective distribution and fusion of 
information—human interpretation will always be critical. 

With the advance of new technologies, the amount of information that can be distributed and 
displayed has grown exponentially. Under these circumstances, poor decision making, slower 
response times, and generally poor performance can result because the individual is too focused 
on processing information rather than performing tasks (Wickens, 2008, 2002). Consequently, a 
major challenge in the field of Human Factors is to convey critical information as efficiently as 
possible (to maximize recipient comprehension and decision making), while simultaneously 
minimizing negative factors such as overload and distraction. 

One prominent method for improving task information delivery is the use of multisensory 
devices, or devices which convey task information through multiple or alternative sensory 
channels instead of the visual channel. Such devices may aid performance by simplifying 
information, maximizing user comprehension, and minimizing user workload. In this report, we 
address the use of tactile cues as a method for improving task feedback, as observed in task 
performance indices. Specifically, we provide a review of studies and a meta-analysis of 
experiments that compared the effectiveness of tactile and visual cues, organized by 
informational complexity. 

1.2 Background 

Although a wealth of experimental research has examined the impact of tactile cues across a 
variety of situations, there is no comprehensive review available to summarize the effect of 
tactile cues on task performance. Recent reviews (Jones and Sarter, 2008; van Erp, 2007) provide 
information with regard to several different tactile and haptic devices, but these reviews limit 
their focus to perception and localization as the primary criteria of variables such as stimulus 
duration, frequency, and intensity. In contrast, our review addresses the effectiveness of tactile 
cues with task performance as the primary criterion. This review also compares the effectiveness 
of tactile cues with that of visual cues (instead of comparing features of the tactors themselves). 
In addition, our work organizes the review of vibrotactile cues around the type of task 
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information being provided (alert, direction, spatial orientation, communication) as a potential 
moderator. What follows is a brief summary of the theoretical background for this review, an 
overview of the study framework, and a discussion highlighting its importance. 

1.3 Attention and Cognition Under High Workload 

There is no doubt that visual displays enable easy and powerful perception, awareness, and 
comprehension. At the same time, researchers have predicted performance gains from the 
presentation of certain types of information using other sensory modalities such as speech, audio, 
haptic, tactile, and three-dimensional (3-D) (e.g., virtual reality, 3-D audio). 

Two theories predict gains from a multisensory display of information. First, Wickens’ Multiple 
Resource Theory (MRT) (2002, 2008) suggests that (1) people have several semi-independent 
cognitive resources; (2) some resources can be used near-simultaneously without detriment to 
performance, while others cannot; (3) tasks requiring the use of different resources can often be 
effectively perfonned together; (4) competition for the same resource can produce interference; 
and (5) dissimilar cognitive resources exist to process information from different sensory 
modalities (e.g., visual, audio, or tactile information). MRT defines these capacities and 
contingencies while predicting the degree to which information from a particular sensory channel 
can be effectively offloaded to another channel. In demonstrating the theory, particular attention 
has been paid to pilot perfonnance (Wickens, Goh, Helleberg, Horrey, and Talleur, 2003) 
although there is nothing unique in that environment that would lead the theory to make 
differential predictions. 

The effectiveness of offloading depends on many factors including: (1) overall workload (e.g., 
offloading is more likely to ease high workload), (2) visual attention requirements (e.g., 
offloading is more likely effective when there are high requirements for both focal and ambient 
vision), (3) cue overlap on subtasks (e.g., visual scanning and tactile/audio direction cues 
effectively combine to support a common task such as target detection or navigation), and (4) the 
degree to which separate tasks demand attention. Wickens (2002) provides extensive discussion 
and examples of these factors in MRT. 

Whereas MRT is focused on aspects of workload and conflict in information processing, the 
Prenav model proposed by van Erp (2007) has a strong emphasis on performance that can be 
automated, such as steering, navigation, and control systems. Prenav explains how tactile cues 
can affect attention, cognition, and perfonnance, with particular regard for the effects of practice, 
automaticity, and intuitive response. Prenav consists of two loops: the infonnation processing 
loop and the workload loop. The infonnation processing loop has the usual li nk s from sensation 
to perception, perception to decision, and decision to action. However, shortcuts from sensation 
and perception to action are hypothesized in order to better account for skill-based performance. 
These shortcuts are related to concepts of automaticity (Shiffrin and Schneider, 1984), skill- 
based learning (Rasmussen, 1983), and naturalistic decision-making (e.g., Klein, 2008; Zsambok 
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and Klein, 1997). These shortcuts are proposed to reduce cognitive overload, thus preserving 
performance quality. 

Within the Prenav model the tactile sense is described as highly intuitive and associated with fast 
reaction times. In fact, studies have found that tactile cues can even be faster than other sensory 
channels, when used for alarms or direction cues. For example, faster braking responses occurred 
in reaction to tactile rear-end collision warnings, compared to when visual cues were used (Scott 
and Gray, 2008). In another context, operators fired more quickly in outdoor target practice using 
targets to the left, right, and center when using torso-mounted tactile direction cues than with 
visual cues (Gilson, Redden, and Elliott, 2007). Tactile cues added to visual cues also yielded 
faster reaction times in simulation-based studies of operator decision-making and performance 
(e.g., Calhoun, Fontejon, Draper, Ruff, and Guilfoos, 2004; Calhoun, Ruff, Draper, and Guilfoos, 
2005; Forster, Cavina-Pratesi, Aglioti, and Berlucchi, 2002). 

1.4 Study Framework: Vibrotactile Cues and Informational Complexity 

MRT and Prenav theories provide the background for the hypotheses and analyses in the current 
review. However, whereas MRT focused on workload and Prenav focused on tasks that are more 
easily automated (e.g., steering and control), we examined the literature by comparing task 
performance of users receiving visual and tactile infonnation cues, organized by informational 
complexity. The framework we employed and associated studies are presented in table 1. 
Preliminary meta-analyses on broad cue comparisons indicated performance benefits from tactile 
cues in general (Prewett et ah, 2006). This review organized experimental studies to further 
distinguish the different types of comparisons made between visual and tactile cues to specify 
tactile cue effectiveness. 

The first analysis includes experiments with a baseline condition and a condition where tactile 
cues were added to the baseline (e.g., baseline visual [B] vs. tactile [T]). In this comparison, the 
tactile cues neither replace nor repeat infonnation provided by any visual cues inherent in the 
task. The second analysis examines studies that compared visual and tactile cue conditions, 
where the visual and tactile cues represented the same information, but only one cue or the other 
is used in experimental conditions (e.g., visual [V] vs. tactile [T]). Finally, the third analysis 
compares conditions with visual cues against conditions with redundant tactile cues are added to 
reinforce the visual information (e.g., visual [V] vs. visual-tactile [VT]). 

Each type of comparison is further distinguished with regard to the level of information 
complexity within cues. A review of the literature reveals numerous vibrotactile research studies 
addressing four primary purposes: tactile cues for alerts, tactile cues for direction, tactile cues for 
orientation, and tactile cues for communication (see table 1). Across these groupings many 
experiments demonstrated that tactile displays were associated with good localization or 
performance but did not have a baseline or visual condition. The studies included in our analyses 
were restricted to those that met criteria necessary for the meta-analysis. 
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Table 1. Organization of meta-analyses. 


Studies included in Baseline Versus Vibrotactile Comparisons by Information Type 

Alerts 

Hammeed et al., 2006; Ho et al., 2007; Hopp et al., 2005; Moorhead et al., 2004; 

Scott and Gray, 2008 

Direction 

Bloomfield and Badler, 2007; Chen and Terrence, 2008; Davis, 2007; 

Dorneich et al., 2006; Elliott et al., 2006; Glumm et al., 2006; Hameed et al., 2007; 
Lindemann et al., 2003; Lindeman et al., 2005; Murray et al., 2003; Tan et al., 2003 

Spatial orientation 

Aretz et al., 2006; Chiasson et al., 2002; Diamond et al., 2002; 

McGrath et al., 2004; Raj et al., 2000; Small et al., 2006; Tripp et al., 2007; 
van Erp et al., 2002; van Erp et al., 2003; van Erp et al., 2006 

Studies included in Visual Versus Visual and Vibrotactile Comparisons by Information Type 

Alerts 

Calhoun et al., 2002; Calhoun et al., 2004; Calhoun et al., 2005; Forster et al., 2002; 
Hopp et al., 2005; Krausman et al., 2007; Moorhead et al., 2004; 

Sklar and Sarter, 1999 

Direction 

Bloomfield and Balder, 2007; Elliott et al., 2007; Latham and Tracy, 2002; 

Tan et al., 2003; van Erp et al., 2002 

Spatial orientation 

Chiasson et al., 2002; Eriksson et al., 2006; McKinley and Tripp, 2007; 
van Erp et al., 2007 

Studies included in Visual Versus Vibrotactile Comparisons by Information Type 

Alerts 

Calhoun et al., 2002; Forster et al., 2002; Hammeed et al., 2007; 

Krausman et al., 2005; Scott and Gray, 2008; Sklar and Sarter, 1999; 
van Erp et al., 2004 

Communication 

Brewster and King, 2005; Ho et al., 2001; Pettitt et al., 2006; 

Direction 

Bloomfield and Badler, 2007; Cholewiak and McGrath, 2006; Davis, 2006; 

Elliott et al., 2006; Elliott et al., 2007; Glumm et al., 2006; Hameed et al., 2006; 
Lindeman et al., 2003; Savick et al., 2008; van Erp et al., 2002; van Erp et al., 2004 

Spatial orientation 

McKinley and Tripp, 2007; Small et al., 2006 


1.5 Tactile Cues for Alerts 

With regard to use as alerts, research participants have successfully perceived and interpreted 
vibrotactile cues in averse, demanding, and distracting situations, such as combat vehicles (e.g., 
Carlander and Errikson, 2006; Krausman and White, 2008), aircrew cockpits (e.g., Diamond, 
Kass, Adrasik, Raj, and Rupert (2002); Rupert, 2000a; 2000b; Rupert, Graithwaite, McGrath, 
Estrada, and Raj, 2004; Sklar and Sarter, 1999), unmanned aerial vehicle (UAV) operation 
(Calhoun, Ruff, Draper, and Guilfoos, 2005; Calhoun, Fontejon, Draper, Ruff, Guilfoos, 2004; 
Calhoun, Draper, Ruff, and Fontejon, 2002), high-speed watercraft (Dobbins and Samways, 
2002); underwater environments (Self, van Erp, Eriksson, and Elliott, 2007), driving (Ho, Reed 
and Spence, 2007; Scott and Gray, 2008), and during strenuous movements (Pettitt, Redden, and 
Carstens, 2006; Krausman and White, 2008). Tactile alerts have been demonstrated effective for 
managing attention during high workload and task switching (Hameed, Ferris, Jayaraman, and 
Sarter, 2006; Ho, Nikolic, and Sarter, 2001; Hopp, Smith, Clegg, and Heggestad, 2005). 

Users have successfully perceived and responded to tactile cues in various body locations. The 
placement of the tactors should correspond with task demands to provide information that easily 
and rapidly recognized. For example, torso-mounted tactors are particularly effective for alerts, 
direction and orientation cues (e.g., Chiasson, McGrath, and Rupert, 2002; Cholewiak and 
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McGrath, 2006; Elliott, Duistermaat, Redden, and van Erp, 2007; van Erp, 2007). Other 
locations that have been successfully used include the wrist (e.g., Dobbins and Samways, 2002; 
Brewster and King, 2005; Calhoun and Draper, 2006; Ho, Nikolic, and Sarter, 2001 ), fingers 
(Hameed, Ferris, Jayaraman, and Sarter, 2007; Lathan and Tracey, 2002), palm of hand (Tang, 
Beebe, and Kramer, 1997), arm (Bloomfield and Badler, 2007), abdomen (Ho, Reed, and 
Spence, 2007; Moorhead, Holmes, and Furnell, 2004), shoulder (Hopp, Smith, Clegg, and 
Heggestad, 2005), and back (e.g., Hameed, Jayaraman, Ballard, and Sarter, 2006; Jones, 

Lockyer, and Piateski, 2006; Lindeman, Yanagida, Sibert, and Lavine, 2003). 

1.6 Tactile Cues for Direction and Spatial Orientation 

Tactile direction and spatial orientation cues have demonstrated faster reaction times, better 
awareness of the task situation, and stable spatial orientation (Gilson, Redden, and Elliott, 2007; 
van Erp, 2007). Direction cues can be simple, composed of two tactors (e.g., Dobbins and 
Samways (2002); Moorhead, Holmes, and Furnell, 2004) or tactors situated within a glove 
(Latham and Tracey 2002), but are more often composed of 8-12-tactor torso belts or linear 
arrangements of tactors in a vest-type garment. They have been demonstrated for navigation in 
cars (Fitch, Kiefer, Kleiner, and Hankey, 2007, van Erp and van Veen, 2004) and simulations 
(Davis, 2006, 2007; Ferris, Penfold, Hammed, and Sarter, 2006; Ferris, Hammed, Penfold, and 
Rao ,2007; Lindeman, Sibert, Mendez-Mendez, Patil, and Phifer, 2005), and for personal land 
navigation (Elliott, Duistennaat, Redden, and van Erp, 2007; Duistennaat, 2005; Duistermaat, 
Elliott, van Erp, and Redden, 2007; Van Erp, Spape, and Van Veen, 2003). Tactile direction cues 
have also been demonstrated for targeting tasks in computer-based simulations (Aretz, Andre, 
Self, and Brenaman (2006; Chen and Terrence, 2007, 2008; Davis, 2006; Glumm, Kehring, and 
White, 2006; McKinley, Gallimore, Lanning, and Simmons, 2005), high-fidelity simulations 
(Eriksson, van Erp, Carlander, Levin, van Veen, and Veltman, 2006; Van Erp, Eriksson, Levin, 
Carlander, Veltman, and Vos, 2007) and vehicles (Carlander and Eriksson, 2006). They have 
also been used in visual search tasks. A common approach to aid visual search uses a vibrotactile 
array that stimulates quadrants on the back that correspond to quadrants on the visual search 
area. Tactors used in this manner have been found to significantly reduce visual search time 
(Brill, Terrence, Downs, Gilson, Hancock, and Mouloua, 2004; Lindeman, Yanagida, Sibert, and 
Lavine, 2003; Tan, Gray, Young, and Traylor, 2003; Young, Hong, and Gray, 2003) and increase 
detection rate (Hameed, Jayaraman, Ballard, and Sarter, 2007). 

Spatial orientation cues, while similar to direction cues, are distinguished based on somewhat 
greater complexity and purpose. Here, the goal is to provide the user with feedback on his or her 
position and location in disorienting environments. There has been particular investment and 
investigation of tactile applications for aircraft and helicopter pilot performance, resulting in 
many demonstrations of effectiveness for aircraft landing and hover tasks. (Chiasson, McGrath, 
and Rupert, 2002; Jansen, Wennemers, Vos, and Groen, 2008; McGrath, Estrada, Graithwaite, 
Raj, and Rupert, 2004; Raj, Kass, and Perry, 2000; Raj, McGrath, Rochlis, Newman, and Rupert, 
1998; Rupert, Graithwaite, McGrath, Estrada, Raj, 2004; Small, Keller, Wickens, Socash, 
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Ronan, and Fisher, 2006; Rupert, 2000a, 200b; Van Erp, Veltman, Van Veen, and Oving, 2003; 
Van Erp, 2005; Van Erp, Groen, Bos, and van Veen, 2006; Van Erp, Veltman, and Van Veen, 
2003). 

1.7 Tactile Cues for Communications 

Tactile patterns (e.g., tactons, tactile melodies) have been shown effective for communications 
(Brewster and King, 2005; Brill and Gilson, 2006; van Veen and van Erp, 2003). This is an 
emerging area of research, so that studies are limited and often focused on localization and 
correct interpretation; there are few studies that compare tactile communications with visual 
communication cues. 

1.8 Hypotheses 

Regarding vibrotactile cue effectiveness and infonnation complexity, MRT and Prenav theories 
yield predictions that guide this review’s hypotheses. MRT emphasizes that multisensory 
displays are more effective when workload is high with multiple non-conflicting tasks. Based on 
this perspective, we expect that the addition of unique or complementary vibrotactile cues will 
facilitate effective task performance across a range of task demands. On the other hand, tactile 
cues that replace visual cues will not provide multisensory cues, but rather replace one modality 
cue for another. Thus, we expect tactile cues to improve performance when added to a baseline 
task or when complementing existing visual cues. We predict that replacement cues, however, 
will not significantly improve perfonnance beyond the level achieved with visual cues. 

HI: Conditions which use complementary vibrotactile cues will have stronger perfonnance 
scores than baseline conditions or visual-only cues (baseline versus tactile [BvT], visual versus 
visual-tactile [VvVT]). 

H2\ Conditions which use vibrotactile cues to replace visual cues will not lead to higher levels of 
performance than conditions that use visual cues only (visual versus tactile [VvT]). 

Prenav predicts that tactile cue effectiveness depends on the type of information provided by 
tactile cues, primarily because certain tactile cues are more intuitive. In particular, this approach 
anticipates benefits in tactile alert, direction, and spatial orientation cues, because they are more 
easily processed by device users. Using the Prenav model, we expect that tactile cue efficacy will 
depend upon the complexity of information being presented. Specifically, we hypothesize that 
conditions using tactile cues for alerts, direction, or spatial orientation will exhibit a stronger 
effect than the more complex communication cues. 

H3 : The advantage of tactile cues over baseline visual cues will be moderated by the 
informational complexity of the cue, in which alerts, direction, and spatial orientation cues will 
exhibit a stronger mean difference statistic than communication cues. 
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1.9 Need for Meta-analysis 

Although existing research has generally indicated that tactile cues benefit performance, much of 
the available evidence is weak when assessed individually. Many studies use small samples in 
which the influence from a single individual could disproportionally impact the outcome. Study 
conclusions are often based exclusively on the /;-value associated with an obtained point 
estimate. It is well known that an obtained /;-value poorly predicts any future obtained /7-value, 
even when the p is obtained from large samples. Gumming (2008), for example, demonstrated 
that if an initial experiment results in two-tailed p = 0.05, the p - interval (e.g., the range of 
/^-values that is likely to occur upon experiment replication) is very large. Using /7-values is also 
a problem for the small sample studies observed in this area, primarily because there is a greater 
chance to obtain a strong effect with a small sample size. With small samples, a single outlier has 
a larger influence on mean differences. This is not the case in larger samples where any 
individual case has a lesser influence on the overall computed statistic (e.g., the F ratio). 
Furthermore, a focus exclusively on statistical significance ignores the value of the effect size. 
For example, two statistically significant (or insignificant) effects may differ by a magnitude that 
yields notable differences in task performance. 

Because of the strong requirement to replicate and validate small study findings with a larger 
sample size, there is a need for a meta-analysis of tactile displays and task performance. Such an 
endeavor addresses the weaknesses of individual experiments by computing an average statistic 
derived from many studies and samples. A meta-analytic review not only serves to validate 
results of individual experimental studies, it also highlights the differences between cue 
complexities and experimental focus that exists between studies (e.g., different independent 
variables). Recent reviews (Jones and Sarter, 2008; van Erp, 2007) provide comprehensive 
information with regard to several kinds of tactile and haptic devices and discuss the 
characteristics that affect perception and localization, such as stimulus type, duration, frequency, 
and intensity. However, they did not address the effectiveness of tactile arrays for task 
performance, nor did they test the impact that informational complexity has on these 
relationships. We argue that the perception of tactor arrays and patterns is not sufficient if the 
cue does not improve task performance. 

The current study addresses the need for a review of the relationships between cue modality, cue 
complexity, and task perfonnance. Our work fulfills this need via meta-analysis of experiment- 
based comparisons of vibrotactile and visual cues in relation to task performance. To further 
clarify the context, these studies are further organized by cue information complexity. 
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2. Method 


2.1 Literature Search 

Multimodal research is being performed in disparate disciplines. As such, key terms (e.g., 
multimodal, audio, tactile, haptic, interface, display) were searched using the following 
databases: Association for Computing Machinery (ACM), Institute of Electrical and Electronics 
Engineers (IEEE), IngentaConnect, Psyclnfo, Web of Science, Defense Technical Information 
Center (DTIC), and Cambridge Scientific Abstracts (CSA). Next, a seven-year retrograde hand- 
search was conducted on the table of contents for the following journals and conference 
proceedings: Human Factors, Human Computer Interaction, Military Psychology, Eurohaptics, 
Society of Industrial and Organizational Psychology, and Institute of Electrical and Electronics 
Engineers. Moreover, the references cited in relevant articles were back-checked to identify 
additional studies. Finally, a number of articles were deemed eligible for inclusion, but lacked 
the statistics (e.g., standard deviations) necessary for meta-analysis. In such cases, the primary 
and/or secondary authors were contacted in an attempt to obtain the requisite statistics for 
inclusion. Study infonnation was entered into a Web-based bibliographic reference database 
(Coovert, Walvoord, Elliott, and Redden, 2008) and is available upon request. 

Inclusion Criteria 

To be included in the meta-analysis, studies had to belong to one of three categories of 
comparisons. In the BvT analysis, tactile cues adding additional information are evaluated 
against a baseline task situation. Examples include adding tactile alerts to a command and 
control decision task or adding tactile direction cues to a simulated driving task. In the VvT 
comparison, tactile cues are evaluated against a condition where visual cues portray the same 
information. This included studies comparing visual versus tactile cues for alerting or navigation 
functions. In this comparison, the tactile cue condition replaces a visual cue with a tactile cue. In 
the VvVT comparison, visual cues are evaluated against the combination of visual and tactile 
cues representing the same information. This is a comparison of single versus multisensory 
display conditions. Only studies with experimentally controlled manipulations (e.g., in which cue 
conditions are assigned) were included. This eliminates the more naturalistic studies in which 
subjects were able to choose which cues to use. We also screened studies for the quality of visual 
and tactile cues. Comparisons were limited to studies that compared cues that were accurate and 
reliable. Thus we excluded some interesting studies that manipulated cue reliability since our 
focus is not on the trust associated with a cue. We also did not include studies of cross-modal 
interactions using cues that were counterintuitive (e.g., where a tactor or visual cue on the left 
was used to cue an event to occur on the right). Although those studies have reason and purpose, 
they are not appropriate for these analyses as our interest is on the effectiveness of the cue itself, 
not on boundary conditions associated with the cue’s use. 
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In addition, eligible studies had to compare modalities in relation to task perfonnance indices 
(e.g., user errors, completion time, and/or reaction time). We did not include perception as an 
outcome nor subjective ratings of workload, preference, or evaluations of device characteristics. 
Studies that used computer mice or joysticks with haptic or tactile features were also not 
included in these comparisons, as they are the focus of a separate meta-analysis. Studies with 
pathological participant populations (e.g., deaf, blind) or with sample sizes of less than four were 
not included. Lastly, studies with the required modalities and outcome variables were excluded if 
we were unable to obtain enough information to calculate an effect size (i.e., information was not 
reported in an article and the authors failed to respond to requests for statistics). 

2.2 Procedure 

Study Coding 

Studies meeting the selection criteria were evaluated on the following dimensions: (1) article 
characteristics (source used to identify the article, type of publication), (2) sample characteristics 
(age, gender, participant population), (3) research design (setting, within vs. between 
comparisons, random assignment, counter-balancing), (4) modality comparisons (BvT, VvT, and 
VvVT), (5) outcome variable (e.g., error rate, completion time, reaction time), and (6) cue 
information complexity (alert, direction, spatial awareness, communication). Ratings were 
recorded for reliability analyses, and initial differences in ratings were resolved through 
consensus meetings. 

A few studies reported multiple outcome measures; for example, a study may have included 
more than one measure of task error. Using multiple effect sizes (due to multiple criteria) in the 
same analysis would violate the assumption of independence. For those cases, the most 
representative single measure was selected for inclusion. If the criterion measures did not differ 
in quality and also represented similar outcomes (e.g., two types of error), the two effect sizes 
were averaged. When studies reported different types of outcomes (e.g., errors and navigation 
time), they were averaged if they were both expected to be influenced by the focal cue of the 
study (visual or tactile). For example, if the visual or tactile cue is providing direction 
information for navigation during target acquisition, the direction information from the tactile 
cue was expected to enable navigation and also allow more visual attention to look for targets. 
Thus, relevant outcome measures could include waypoint completion, navigation time, and target 
detection rate. In this way, multiple measures comprised the combined effect size estimate when 
appropriate. 

Reliability Analyses 

For categorical variables (modality comparison, outcome variable, task level, task type, and 
workload), a kappa coefficient was computed to assess reliability. Coefficients ranged from 0.84 
(outcome) to 0.90 (modality comparison), indicating acceptable reliability levels for the ratings. 
Initial rater consistency on effect sizes was modest, due to the many ways an effect size could be 
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obtained from the study (e.g., due to numerous outcomes). Thus, an intraclass correlation 
coefficient using two of five random raters (ICC 2 , 5 ) (Shrout and Fleiss, 1979) was calculated 
after the consensus meeting. Inter-rater reliability is 0.92 for effect size ratings. 

Calculation of Statistics 

A random effects model (Hunter and Schmidt, 2004) was used for the analyses because variance 
was assumed to stem from study characteristics as well as sampling error, whereas the fixed 
effects model only assumes sampling error. As a result, random effects models attribute less 
variance to the treatment effect than fixed models, resulting in wider confidence intervals and a 
more conservative test. Analyses were conducted using the procedures of Hedges and Olkin 
(1985) as applied in Comprehensive Meta-analysis (CMA) 2.0 (Borenstein, Hedges, Higgins, 
and Rothstein, 2005). This procedure weights studies by the inverse of their error variance, such 
that studies with higher standard errors for their test statistic receive lower weights. Analyses 
yielded Hedge’s g, which is preferred in that it corrects for a small-sample bias where the 
standard error (SE) is overestimated (Hedges, 1981). For within-groups studies in which paired 
correlation data were not available, we estimated a correlation of 0.3 to 0.5, using the more 
conservative estimate if there was a difference in overall effect size. Since they are not relevant 
here, we did not correct for measurement artifacts (e.g., for restriction of range, unreliability), 
which would be done if the focus of our study were an issue such as test validation for personnel 
selection (Hunter and Schmidt, 2004). 

The Hedges and Olkin (1985) procedure also estimates the amount of variance among the 
included studies that exceeds what one would expect from sampling error alone (the random 
effects variance component [REVC]). Whereas the REVC is a technical number used in meta- 
analytic calculations, the percentage of total variance that cannot be accounted for by sampling 
error is more intuitively presented as I . We also report the SE associated with Hedge’s g, as well 
as 95% confidence and credibility intervals. Whereas confidence intervals present the precision 
and possible range of the meta-analytic statistic, credibility intervals reflect the expected 
distribution of effect sizes within the population of studies. Confidence intervals are calculated 
from Hedge’s g and the SE; credibility intervals are calculated from g and the REVC. 

Moderator analyses were conducted using the analog to analysis of variance (ANOVA) 
procedure (Hedges and Olkin, 1985). This procedure treats studies in different coding categories 
as independent groups and detennines if their effects are statistically different. Significant study 
heterogeneity (Q to tai) in overall analyses indicates a possibility that the overall effect is impacted 
by a moderator variable. Chi square tests then compare within-group variance to between- 
subgroup variance (Qb), yielding a test of moderation. 

Finally, in any meta-analysis there is the threat of publication bias, whereby the sample of 
studies used in the analysis is skewed due to the publication process. Statistically insignificant 
studies are less likely to be pursued or published and tend to be under-represented in meta- 
analytic samples. To detect publication bias, we created funnel plots to graphically observe the 
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distribution of studies by sample and effect size. The presence of publication bias is detected by 
an asymmetric graph, in which small-sample studies only co-occur with large effect sizes, but 
large-sample studies contain both small and large effects (for a review, see Light and Pillemer, 
1984; Hunter and Schmidt, 2004, pp. 501). In addition to funnel plots, a classic fail-safe N can be 
calculated. This statistic estimates the number of missing null studies (g = 0) needed to turn the 
significant results found in a meta-analysis into an insignificant one (Rosenthal, 1984). 

In sum, the funnel plots provides a visual depiction of the relationship between sample size and 
reported effect size in order to reveal if a bias exists in reported studies. The fail-safe N tells how 
many studies with non-significant results would have to be added to a significant meta-analytic 
finding to turn it into a non-significant one. Combined, the information provided by the funnel 
plots and the fail-safe N allows us to gauge the confidence of our findings. Due to space 
constraints we do not provide the funnel plots as they collaborate the reported fail-safe N. 


3. Results 


Table 2 presents the meta-analytic statistics (described previously) for our work. In general, 
tactile cues lead to increased task performance across all three comparisons. We now present 
results for meta-analyses associated with each hypothesis. 
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Table 2. Meta-analytic results for vibrotactile cue effectiveness. 


Comparison 

k 

g 

SE 

95% Cl 

95% CV 

REVC 

I 2 

Qtotal 

Qb 

Fail-safe N 

Overall Analyses 











BvT 

26 

1.15“ 

0.21 

(0.74, 1.56) 

(0.02, 2.28) 

0.335 

78.10 

114.16" 

0.31 

1709 

VvT 

23 

-0.95 

0.22 

(-0.41, 2.32) 

(-1.02,2.92) 

1.011 

92.02 

288.09" 

23.13" 

198 

VvVT 

17 

0.89** 

0.18 

(0.53, 1.24) 

(-0.01, 1.79) 

0.210 

65.49 

46.36" 

1.63 

431 

Analyses x Cue info 











BvT 








114.16" 

0.31 


Alarms 

5 

1.60** 

0.36 

(0.90, 2.29) 

(-0.60, 3.80) 

1.255 

90.33 

41.38" 

— 

89 

Direction 

11 

0.91** 

0.21 

(0.50, 1.32) 

(0.25, 1.57) 

0.114 

58.22 

23.93" 

— 

279 

Spatial Orientation 

10 

1.13** 

0.24 

(0.66, 1.60) 

(-0.24, 2.50) 

0.490 

81.46 

48.53" 

— 

215 

VvT 








288.09" 

23.13" 


Alarms 

7 

1.81“ 

0.45 

(0.92, 2.69) 

(-2.15,3.05) 

1.760 

90.58 

63.71" 

— 

99 

Direction 

11 

-0.20 

0.29 

(-0.78, 0.37) 

(-1.80, 1.40) 

0.665 

90.04 

110.46" 

— 

— 

Communication 

3 

2.29" 

0.61 

(1.09,3.49) 

(-0.74, 5.32) 

2.390 

95.00 

39.98" 

— 

63 

Spatial Orientation 

2 

-0.02 

0.70 

(-1.40, 1.36) 

(-0.02, -0.02) 

0.000 

0.00 

0.31 

— 

— 

VvVT 








46.36" 

1.63 


Alarms 

8 

1.08" 

0.24 

(0.62, 1.55) 

(0.02,2.14) 

0.294 

68.47 

22.20** 

— 

124 

Direction 

5 

0.64* 

0.26 

(0.12, 1.15) 

(0.19, 1.10) 

0.054 

41.41 

6.83 

— 

24 

Spatial Orientation 

4 

0.93" 

0.40 

(0.27, 1.60) 

(-0.68,2.54) 

0.674 

78.28 

13.82" 

— 

15 


Note. B=Baseline, V=Visual, T=Tactile. k = number of studies; g = weighted mean difference; SE = Standard error of g; * p<0.05, ** p<0.01; Cl = Confidence 
Interval; CV = Credibility Interval; REVC = random effects variance component; T = percentage of between-study variance that is not sampling error; Q tota iand 
Q b reflect total variance and the between-category variance, respectively; Fail-safe N is the number of studies where g =0 that is needed to make g insignificant. 



3.1 Baseline versus Tactile 

The comparison between tactile cues and a baseline condition was the most common one found 
in the literature. Studies of this type are relatively homogenous in characteristics. Typical 
examples are provided by Elliott and colleagues (2006) and Domeich et al. (2006), whose work 
examined actual land navigation (and secondary tasks) in the field. Most other studies were 
based on simulation and virtual reality (VR) based situations. Performance measures varied, but 
most reflected some type of speeded perfonnance (e.g., reaction time, time to navigate, number 
of targets or icons found in speeded situations, and so forth). 

Hypothesis 1 stated that adding tactile cues to a baseline would improve task performance. This 
hypothesis was generally supported as nearly all studies favored the tactile condition. Overall, 
Hedge’s g was 1.15(SE = 0.21,/?<0.01). Although nearly all studies reported a positive effect 
in tactile cues, there was significant variation in study effect sizes (Q to tai = 114.16 ,p< 0.01), 
indicating the potential for moderators. 

Moderator analyses by cue complexity, however, were not significant ( Q h = 0.31, n.s.). Each 
type of tactile cue provided a significant benefit to tactile performance compared to baseline 
conditions: alerts (g = 1.60; SE = 0.36, p < 0.01), direction cues (g = 0.91, SE = 0.21, p < 0.01), 
and spatial orientation cues (g = 1.13, SE = 0.24 ,p < 0.01). 

We can have a good deal of confidence in these findings. Although the funnel plot dispersion 
indicated more small-sample studies with positive effect for tactile cues, the fail-safe N reveals 
that it would take 1709 “missing” studies with null findings to change the obtained results. 

3.2 Visual versus Tactile 

Visual and tactile alerts or direction cues were compared for a variety of tasks, including simple 
reaction time or visual search (cf., Forster et al., 2002), simulated driving and/or targeting tasks 
(e.g., Davis, 2006), complex cockpit, UAV, or command simulations (e.g., Sklar and Sarter, 
1999), as well as communicating information (e.g., Redden, Carstens, Turner, and Elliott, 2007), 
localizing from dense multi-tactor displays (Cholewiak and McGrath, 2006), land navigation in 
the field (e.g., Elliott et al., 2007), and orienting in virtual environments (Bloomfield and Badler, 
2007). 

Hypothesis 2 stated that tactile cues would not be effective when replacing visual cues and was 
partially supported. Hedge’s g was not significant for this comparison, primarily due to the 
conservative test of the random effects model. Although the effect of replacing visual cues with 
tactile cues was large (g = 0.95, SE = 0.22), significant variation (Q to tai = 288.89, p < 0.01) 
prevented a statistically significant result. We note that a more liberal, fixed effects model would 
find this mean difference statistically significant. The effectiveness of replacing visual cues with 
tactile appears to be highly dependent upon other experimental factors, as judged by the strong 
variation between individual studies. 
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Moderator analyses by cue complexity revealed a significant interaction (Qb = 23.13 ,/? < 0.01). 
Examination by type reveals tactile cues for alerts (g = 1.81, SE = 0.45,/? < 0.01) and 
communication (g = 2.29, SE = 0.61 ,P< o .01) lead to stronger perfonnance; this is not the case 
for cues relative to direction (g = -0.20, SE = 0.29, n.s.) and spatial orientation (g = -0.02, SE = 
0.70, n.s.). 

Hypotheses 3 specified alert, direction, and spatial orientation cues would aid performance more 
so than communication cues. Results were consistent with predictions for alerts, but not for other 
types of cues. Inspection of the funnel plot is satisfactory and the classic fail-safe N computes to 
198, meaning it would take 198 studies with non-significant findings to change our results. 

3.3 Visual versus Visual-Tactile 

VvVT studies ranged from simple reaction time or tracking tasks (e.g., Moorhead et ah, 2004) to 
more complex demands such as communications (e.g., Bloomfield and Badler, 2007), driving 
(e.g., van Erp et al., 2002), cockpit or command, control, and communications (C3) simulations 
(e.g., Calhoun et al., 2004), land navigation (Elliott, 2007), teleoperation in VR environments 
(Lathan and Tracey, 2002), advanced tactile arrays for spatial orientation in flight (e.g., Chiasson 
et al., 2002), or localization of a dense array of tactors (Tan et al., 2003). When comparing visual 
only cues to complementary visual and tactile cues, we predicted a difference between conditions 
in favor of multisensory conditions (Hypothesis 1). This prediction was strongly supported as 
nearly all studies demonstrated a positive effect (g = 0.89, SE = 0.18,/? < 0.01). There was also 
significant (though small) variation in the effect magnitude (Q to tai = 46.36,/? < 0.01), suggesting 
that cue complexity may yet moderate this comparison. 

Re-analyses based on cue complexity revealed a non-significant moderation effect (Qb= 1-63, 
n.s.). Direction cues produced a marginally weaker effect size (g = 0.64, SE = .26, /? < 0.05) than 
alert (g= 1.08, SE = .24,/? < 0.01) or spatial orientation cues (g = .93, SE = .40,/? < 0.01). 
However, these differences were not strong enough to confirm a significant interaction by cue 
complexity. 

Inspection of the funnel plot suggested some heteroscedastic dispersion. However, the classic 
fail-safe N is 431, signaling a large number of “missing” studies with non-significant results 
would be needed to reverse the conclusion that adding tactile information to visual will increase 
performance significantly beyond that found in a visual-only system. 

3.4 Overall Summary of Results 

When added to a baseline task to existing visual cues, the addition of tactile cues enhanced task 
performance. Using tactile to replace visual cues produced mixed results that were moderated by 
cue information complexity. Tactile alerts were effective when replacing visual alerts, but tactile 
direction cues did not improve perfonnance when replacing visual direction cues. 
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4. Discussion of the Effectiveness of Tactile Cues 


This review addressed a need to understand when, why, and how tactile cues enhance task 
performance. We first discussed two theoretical approaches (Wickens’ MRT, van Erp’s Prenav) 
to tactile cues and then provided a quantitative review of the literature to detennine empirical 
performance benefits. To conduct our review, we developed a framework that organized studies 
by the type of comparison and by the complexity of information conveyed in the cue. We 
predicted that tactile cues would be more effective when adding unique task information (e.g., to 
a baseline) or when adding redundant task cues (e.g., added to a visual cue). Conversely, we 
expected a smaller influence when tactile cues were used to replace visual ones. Study results 
generally confirmed these predictions. In addition, we expected that tactile cues would be more 
effective in intuitive applications (e.g., cues for alerts, direction, and orientation). Study results 
partially confirmed this expectation, as cue complexity moderated the relationship between 
visual and tactile cues as replacements (VvT), but this effect was not observed in other 
comparisons. Specifically, tactile cues effectively replaced visual cues when providing alerts and 
communications, but they did not provide an advantage when giving information on direction or 
spatial orientation. 

4.1 Adding Tactile Cues to Support Performance 

When tactile cues were added to a baseline task, all subgroup analyses were significant, 
including alert, direction, and spatial orientation cues. There were no studies in this comparison 
category that used tactile cues for communication cues. Tactile cues are found to be very 
effective when used to manage attention and to support tasks such as visual search, navigation, 
driving, target acquisition, and piloting. 

4.2 Use of Multisensory Cues 

Tactile cues are also effective when used in a multisensory presentation. Comparisons from this 
review suggest using multisensory over visual cues to convey task infonnation. Results when 
using tactile infonnation as alerts favor tactile cues, with only one study having a confidence 
interval that included zero (indicating the results are not conclusive). Subgroup analysis for 
directional cues also significantly favors the multisensory condition. Finally, studies comparing 
VvVT cues for orientation also favored a multisensory presentation. 

4.3 Replacing Visual Cues with Tactile Cues 

Results for studies that compared a tactile to a visual presentation of the same information 
indicate considerable variation among studies. Studies using tactile cues as alerts or 
communication tended to support use of tactile cues. However, while tactile communications 
were favored, it must be noted that they were contrasted with hand and arm signals (vs. verbal 
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communications). Thus, additional research is needed that compares the effectiveness of tactile 
communication with verbal/audio communication. 

In contrast, studies using tactile direction or spatial orientation cues to replace visual ones did not 
yield a significant effect, though few studies examined spatial cues. It is worth noting that the 
benefit of replacing visual direction cues with tactile direction cues varied significantly between 
studies. 

An examination of studies in which tactile cues did not enhance performance fails to indicate a 
single explanation, but rather suggests a number of situational factors that might influence 
results. Consider the fact that a rich variety of direction cues occur in experiments, yet were 
combined in this meta-analysis. Glumm et al. (2006), for instance, examined effectiveness of a 
torso-based tactile array that provided targeting information in a PC-based single screen 
simulation. The visual icon was within operator view at all times and exactly matched the 
targeting control device in appearance. Thus, the visual cue was easily seen, always in view, and 
intuitively comprehended. Consequently, operators performed very well with the visual cues. In 
another example, Davis (2006) compared a torso tactile belt to visual icons in a PC-based single 
screen simulation. The visual icon was also readily seen, always in view, very easily 
comprehended (i.e., a moving map), and was also associated with better perfonnance. In 
contrast, Savick et al. (2008) and Krausman et al. (2005) used PC-simulations, but required the 
operator to look at multiple screens. This forced operators to divide attention, searching for 
incoming information on yet additional screens. In these situations, the tactile cues were more 
effective as an alert and a directional cue. These contrasting results are entirely consistent with 
MRT theory, in that the tactile cues were more effective when workload was higher and 
comprised multiple tasks needing a high degree of attention management. 

Another set of conflicting results is found when tactile cues support land navigation. Elliott et al. 
(2006) documented a positive relationship between the tactile presentation of navigation 
(direction) information and task performance. Conversely, a follow-up study (Elliott et al., 2007) 
found that visual and multisensory cues were associated with higher performance when the 
visual cue was more easily comprehended. The visual display used in the latter study was very 
simple, using an intuitive arrow display from a commercial handheld global positioning system 
(GPS), compared to the more effortful standard Army alphanumeric display used in the fonner 
study. Other differences between studies include those commonly found in the field 
environment, such as participants navigating in daylight conditions versus night operations, good 
versus adverse weather conditions, and so forth. Another factor that may have affected 
navigation time with the tactile belt is operator familiarity and trust. Although participants 
rapidly learned how to use the tactile belt and were told they could not “outrun” the signals, they 
reported that they tended to increase speed only after they had used the system for a while. 
Similarly, ratings of the operational effectiveness of the tactile belt increased significantly 
between pre-experiment and post-experiment assessments. In summary, a closer examination of 
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studies suggest that tactile cues support waypoint land navigation, particularly when visibility is 
low or attention must be focused on local terrain (e.g., obstacles, dangerous terrain). 


5. Conclusions 


5.1 Strength of Findings 

Use of Hedge’s procedure requires complete data from primary studies, such as the standard 
error of the statistic, the study effect sizes (mean difference), and sample size. One might assume 
this is a limitation to our meta-analysis because many studies that met preliminary experimental 
criteria did not report complete data for calculating Hedge’s g and the SE. Also, indicators of 
variance or confidence intervals were sometimes not reported. In addition, F tests were reported 
for multiple groups, but there is no reference to planned or follow-up paired comparisons, and 
detailed results (means, variance, t or p values) are often not reported for non-significant 
comparisons. We argue, however, that using a conservative set of criteria for article selection in 
meta-analysis will yield a solid foundation of articles from which to conduct the meta-analysis. 
As a result, because the data employed for the meta-analysis has been thoroughly examined and 
vetted, our findings and conclusions are veridical and conservative. 

The results of every significant meta-analysis should be examined via the funnel plot and classic 
fail-safe N to ensure the conclusions are warranted. In our case, the funnel plots are all 
reasonable, and the aggregated effect size for each comparison is high. Classic fail-safe N tests 
for each comparison indicate that the results are robust, as the statistic ranged from several 
hundred to nearly two thousand non-significant studies that would have to be added to the meta¬ 
analysis in order to change our conclusions. 

5.2 Future Research 

Although our conclusions provide a solid foundation upon which to build, there was still 
significant variation in study comparisons that were not accounted for by cue complexity (as 
indicated by the heterogeneity statistics). Further research in this area should clarify moderating 
factors and refine guiding principles to detennine when, where, why, and how to best use tactile 
cues in support of operator performance in demanding or complex environments. For example, 
research should ascertain the effects on levels of performance complexity, from simple reaction 
times to complex decision making under uncertainty. Workload was ultimately excluded from 
our review because very few studies manipulated or measured subjective workload in a 
systematic manner. Future studies also need to further test and refine theory-driven predictions 
with regard to overall workload and demands for attention management. Additional experiments 
should examine individual differences, particularly with regard to attention management skills, 
task experience, and automaticity. Attention management and other cognitive skills can be 
assessed via PC-based cognitive batteries such as the Automated Neuropsychological 
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Assessment Metrics (ANAM) (Reeves, Winter, Kane, Elsmore, and Bleiberg, 2001) and the 
Synwin task (Elsinore, 1994). 

To date, experiments have focused on short-term performance, in which tasks are performed by 
novices or are associated with short periods of training. There is reason to believe that “ease of 
comprehension” is a critical factor in the design of any perceptual cue, be it visual, audio, or 
tactile. This begs several questions. To what degree will initial differences among cue displays 
last? Can such differences be overcome or reversed with extended practice? Research comparing 
alternate and multiple modalities of cue information should be performed in task situations 
where these factors can be effectively manipulated and controlled; these designs would further 
predict and model the impact of levels of task demand and cue complexity. Finally, we must 
consider effects across different task demand situations, from stationary control operators to 
expert first responders in an emergency environment. Principles cannot be applied regardless of 
situation. Instead, designers must always consider in detail the specific cognitive and situational 
demands, in order to best determine where bottlenecks occur and how they can be alleviated. 

In summary, we provide a comprehensive analysis of the impact of vibrotactile cues on task 
performance. In order to distinguish different effects of tactile cues, we differentiated between 
comparison types and cue complexity in our analyses. Results demonstrate positive evidence for 
the effectiveness of tactile cues, but benefits are not as distinct when tactile cues replace, rather 
than complement, visual cues. Results are also consistently positive for multisensory 
applications. The variation among studies indicates the need for careful consideration of task 
demands and cue characteristics by interface designers. 
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